1. Field of the Invention
The present invention relates to an image recovery processing technology that uses an image recovery filter.
2. Description of the Related Art
Various methods have been proposed for performing correction processing on image data. When generating image data by capturing an object with a digital camera, the obtained image data is at least partially deteriorated due to aberration in the imaging optical system.
A blur component in image data is a factor in optical system spherical aberration, coma aberration, curvature of field, astigmatism and the like. When an ideal optical system that is free of aberration and does not suffer any effects of diffraction is used, the light beams coming from a single point of the object that should reconverge at a single point on the imaging plane in fact form an image in which the light beams do not converge at a single point, and are spread out. The distribution of these spread out light beams is the blur component. Although in optics this is referred to as “point spread function” (PSF), for image data we will refer to this as “blur component”. Further, although the term blur component of image data can also refer to blurred image data that is not in focus, for example, here this term is used to refer to blurring due to the effects of aberration in the optical system even when the image is in focus. Further, the fact that optical system axial chromatic aberration, color spherical aberration and color coma aberration are factors can be said to be due to differences in the blur component at each light wavelength.
An example of a known method to correct deterioration of the blur component in image data is to use information about the imaging optical system optical transfer function (OTF). Although this method is referred to using names such as image recovery and image restoration, in the following description processing that corrects deterioration of image data using information about OTF of the imaging optical system will be referred to as image recovery processing.
An outline of image recovery processing will now be described. When deteriorated image data is represented as g(x,y), the original image data as f(x,y), and a PSF obtained from an inverse Fourier transform on the optical transfer function as h(x,y), the following equation holds.g(x,y)=h(x,y)*f(x,y)Where * represents convolution and (x,y) represents image data coordinates.
Further, if the equation is subjected to a Fourier transform and converted into a display format at a frequency plane, the resultant equation is in the form of an integral for each frequency, as illustrated below. In this equation, since H is obtained by subjecting a PSF to a Fourier transform, H is an OTF. Further, (u,v) represents coordinates in a two-dimensional frequency plane, i.e., represents the frequency.G(u,v)=H(u,v)·F(u,v)
To obtain the original image data from the deteriorated image data, first, both sides are divided by H, as follows.G(u,v)/H(u,v)=F(u,v)
Then, the original image data f (x, y) is obtained as a recovered image by subjecting the F (u,v) to an inverse Fourier transform to return to the actual plane.
If the value obtained by subjecting 1/H in the above formula to an inverse Fourier transform is R, the original image data can similarly be obtained by performing convolution processing on the image data in the actual plane as illustrated in the following equation.g(x,y)*R(x,y)=f(x,y)
This R (x, y) is called an image recovery filter. Since the OTF varies according to the imaging state, such as the state of the zoom position and the state of the aperture diameter, the image recovery filter to be used in image recovery processing also needs to be changed according to the OTF. For example, Japanese Patent Application Laid-Open No. 10-165365 discusses a technique for resolving image blur by using a PSF corresponding to a used fluorescence wavelength for a range that is outside of the focus range of the imaging means in an endoscope for observing inside the body. Since the fluorescence is faint, an object optical system having a small F number is needed. Since the focal depth becomes more shallow if an object optical system having a small F number is used, this technique is trying to obtain an in-focus image by performing image recovery processing on a range that is out of focus.
As described above, image quality can be improved by performing image recovery processing on captured image data and correcting various aberrations. However, in actual imaging, the imaging state of the image data and the state of the image recovery filter for recovery of the image data may not completely match. An example of such a case is when there is a saturated pixel in the captured image data. Since a saturated pixel has lost its original object information, a situation arises in which the deterioration state of the image data to be actually obtained and the deterioration state of the image data that is envisaged as the recovery target by the image recovery filter do not match.
Further, when selecting or generating an image recovery filter according to imaging information such as the lens focal length, the aperture value, and the imaging distance, a difference can occur between the actual imaging state and the imaging information to be used. A situation in which the deterioration state of the image data to be actually obtained and the deterioration state of the image data that is envisaged as the recovery target by the image recovery filter do not match tends to occur especially for imaging distance, because the object distance differs depending on the angle of view when capturing a three-dimensional object.
In particular, when the actually obtained image data is sharper than the image data envisaged as the recovery target by the image recovery filter, if the image recovery filter is used, the recovered image data becomes over-recovered, which causes image deterioration such as undershoot and overshoot at the edge portions. In particular, undershoot at a low luminance portion is amplified by the gamma correction processing that is applied after image recovery processing, so that the resultant image data looks strange.
Further, even in the technique discussed in Japanese Patent Application Laid-Open No. 10-165365, when there is a difference between the envisaged PSF and the actually obtained image data, over recovery occurs, which can be thought to lead to deterioration in the quality of the recovered image data.